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Abstract 


Many  Department  of  Defense  (DoD)  information  technology  (IT)  programs  are  plagued 
by  immense  bureaucracy,  seemingly  endless  documentation,  cost  overruns,  and  poorly  defined 
requirements  that  cause  considerable  delays  in  getting  needed  capabilities  to  the  field.  To 
facilitate  the  more  efficient  and  timely  delivery  of  capabilities  to  the  warfighter,  the  DoD  has 
emphasized  finding  ways  to  improve  the  acquisition  of  capabilities  for  the  warfighter.  One 
strategy  is  Agile  software  development.  DoD’s  interest  in  Agile  prompted  policy  updates  and 
initiatives  such  as  Better  Buying  Power  that  accentuate  innovation,  speed,  and  elimination  of 
bureaucratic  processes.  However,  lack  of  agility  is  still  identified  today  as  a  persistent  issue.  This 
study  examined  the  DoD  acquisition  framework  to  determine  the  extent  to  which  it  facilitates  or 
hinders  Agile  IT  software  development.  The  qualitative  investigation  included  a  historical  review 
of  literature  from  industry  and  government.  The  review  revealed  six  areas  specific  to  the  DoD 
acquisition  process  that  presented  challenges  and  constraints:  acquisition  oversight,  contracting, 
cost  estimation,  information  assurance,  program  cost  and  performance  monitoring,  and 
requirements  management.  Additionally,  culture  was  highlighted  by  the  Software  Engineering 
Institute  and  others  as  a  principal  barrier  to  adopting  Agile  in  the  DoD.  Agile  should  not  be 
considered  a  blanket  solution  for  all  DoD  IT  programs.  However,  as  MITRE  asserted,  it  is  a 
viable  option  for  programs  able  to  streamline  their  organizational  structures  to  accommodate  a 
process  that  emphasizes  smaller,  more  frequent  capability  releases. 
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Chapter  1  -  Introduction 


Background 

Historically,  Department  of  Defense  (DoD)  programs  have  not  generally  been  regarded 
as  efficient,  streamlined  efforts  that  deliver  timely  capabilities  to  the  warfighter.  Many  programs 
are  plagued  by  immense  bureaucracy,  seemingly  endless  documentation,  cost  overruns,  and 
poorly  defined  requirements  that  cause  considerable  delays  in  getting  needed  capabilities  to  the 
field.  In  a  Defense  Science  Board  (DSB,  2009)  report  tasked  with  examining  DoD  policies  and 
procedures  for  information  technology  (IT)  acquisition,  DSB  Chairman  William  Schneider,  Jr. 
included  a  memorandum  to  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and 
Logistics.  In  it,  he  wrote,  “The  fundamental  problem  DoD  faces  is  that  the  deliberate  process 
through  which  weapon  systems  and  information  technology  are  acquired  does  not  match  the 
speed  at  which  new  IT  capabilities  are  being  introduced  in  today’s  information  age. 
Consequently,  the  principal  recommendation  of  the  study  is  that  the  Department  needs  a  new 
acquisition  system  for  information  technology”  (para.  2). 

The  DoD  is  subject  to  a  changing  operational  environment,  budgetary  constraints,  the 
rapid  evolution  of  technology  that  can  accelerate  obsolescence,  and  an  increasing  dependency  on 
software  in  both  weapon  and  business  systems.  Due  to  these  circumstances,  the  DoD  has  placed 
additional  emphasis  on  finding  ways  to  improve  the  acquisition  of  capabilities  for  the  warfighter. 
One  strategy,  which  some  DoD  organizations  have  successfully  implemented,  and  which  has 
been  used  in  the  commercial  sector  for  decades,  is  Agile  software  development.  Several 
organizations  within  DoD  have  initiated  efforts  to  integrate  Agile  concepts  into  the  DoD 
acquisition  framework  (which  is  defined  here  as  the  laws,  regulations,  policies,  and  guidance  that 
govern  the  acquisition  process).  For  example,  in  2012,  DoD  Chief  Information  Officer  Teri 
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Takai  provided  a  10-point  information  technology  modernization  plan.  Ms.  Takai  specified  Point 
Four  of  the  plan  as  enabling  Agile  IT  through  active  user  involvement  and  more  frequent 
delivery  of  usable  capabilities.  DoD  Instruction  (DoDI)  5000.02  was  also  updated  to  include 
allowances  for  tailoring  acquisition  programs  to  support  increased  agility  in  delivering 
capabilities  (DoD,  2017). 

What  Is  Agile?  There  are  many  characterizations  of  the  term  “agile”  within  the  context 
of  the  DoD.  For  the  purposes  of  this  paper,  Agile  is  defined  from  the  perspective  of  IT  software 
development.  Agile  (big  “A”)  is  the  ability  to  produce  and  react  to  change,  enabling  success  even 
in  an  environment  of  uncertainty  and  volatility.  Agile  software  development  is  an  overarching 
term  for  a  set  of  practices  and  procedures  for  developing  software.  The  Agile  methodology 
enables  requirements  and  solutions  to  evolve  through  the  process  of  stakeholder  collaboration, 
and  it  is  characterized  by  functionally  diverse,  self-organizing  teams  (Agile  Alliance,  2001). 
Simply  put,  Agile  software  development  is  a  means  to  produce  software  in  a  more  collaborative 
manner,  in  which  the  stakeholder  is  engaged  throughout  the  entire  development  process.  This 
constant  communication  and  level  of  engagement  is  meant  to  ensure  that  the  capabilities  being 
produced  are  consistent  with  stakeholder  needs,  which  can  be  refined  over  time.  Agile  also 
allows  higher  priority  requirements  to  be  addressed  first  and  enables  more  frequent  delivery  of 
useful  capability  based  on  stakeholder  feedback. 

Problem  Statement 

DoD’s  interest  in  Agile  software  development  prompted  updates  to  some  policies  and  the 
creation  of  initiatives  that  emphasize  innovation,  speed,  and  elimination  of  bureaucratic 
processes  to  deliver  capabilities  more  rapidly  to  the  warfighter.  Several  government 
organizations  and  industry  partners  have  conducted  studies  on  the  suitability  of  the  Agile 
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methodology  in  the  DoD  environment.  Some  of  the  studies  provided  recommendations  and  best 
practices  for  adopting  Agile  approaches  within  the  DoD.  However,  lack  of  agility  in  the  defense 
acquisition  environment  is  still  identified  as  a  persistent  issue.  The  National  Defense 
Authorization  Act  (NDAA),  which  specifies  the  DoD’s  budget  and  authorized  expenditures, 
provides  Congress’s  perspective  on  acquisition  reform:  “the  conventional  acquisition  process  is 
not  sufficiently  agile  to  support  warfighter  demands. .  .the  current  process  is  so  rigid  and  time- 
consuming  that  the  Department  is  often  unable  to  effectively  tap  into  the  innovation  occurring  in 
the  commercial  marketplace”  (Conference  Report  to  Accompany  H.R.  1735,  2015,  p.  153  of 
“Division  A — Department  of  Defense  Authorizations”). 

Agile  software  development  has  been  recognized  within  the  DoD  as  a  viable  means  to 
improve  and  expedite  the  delivery  of  IT  capabilities  to  the  warfighter.  Nevertheless,  conditions 
may  still  exist  that  are  impeding  further  adoption  of  Agile  practices  in  the  DoD  environment. 
This  study  asserts  that  the  DoD  acquisition  process  is  not  properly  structured  for  adoption  of 
Agile  software  development. 

Purpose  of  This  Study 

This  study  was  intended  to  explore  potential  barriers  in  the  DoD  IT  acquisition 
environment  that  may  have  obstructed  greater  Agile  adoption  across  the  DoD.  Specifically,  this 
study  investigated  current  policies,  processes,  guidance,  and  governing  laws  and  regulations  to 
determine  the  extent  to  which  they  facilitate  or  hinder  the  execution  of  Agile  software 
development  for  acquisition  information  systems.  The  study  considered  both  government  and 
industry  perspectives  on  the  applicability,  as  well  as  any  challenges,  of  Agile  software 
development  in  the  DoD  environment. 
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Significance  of  This  Research 

Tangible  effort  has  been  invested  in  acquisition  reform  and  implementation  of  Agile 
initiatives  within  DoD.  Previous  research,  such  as  that  conducted  by  the  Software  Engineering 
Institute,  has  disputed  the  claim  that  DoD  policy  precludes  the  use  of  Agile  (Lapham,  Williams, 
Hammons,  Burton,  &  Schenker,  2010).  In  fact,  many  DoD  organizations  have  successfully 
implemented  Agile  processes  to  execute  software  development  within  their  programs. 

However,  despite  these  efforts,  agility  is  still  highlighted  as  a  significant  challenge  in 
today’s  environment.  DoD  programs  are  still  challenged  with  delivering  timely  capabilities.  This 
study  synthesized  numerous  perspectives  across  government  and  industry  to  identify  current 
roadblocks  that  continue  to  impede  the  successful  implementation  of  Agile  in  the  DoD.  The 
ultimate  goal  of  this  study  is  to  promote  further  discussion  about  the  effectiveness  of  acquisition 
reform  efforts  to  date  and  about  what  can  be  done  to  streamline  the  current  DoD  acquisition 
framework  to  enable  greater  adoption  of  Agile  methodologies. 

Overview  of  the  Research  Methodology 

The  strategy  for  this  study  was  to  perform  a  qualitative  examination  of  available 
historical  literature  from  both  industry  and  government  sources  regarding  Agile  implementation 
in  the  DoD  environment.  This  data  was  then  further  dissected  to  identify  commonalities  and 
differences  in  perspective  to  find  potential  focus  areas  within  the  acquisition  framework  that 
continue  to  create  challenges  for  DoD  adopters.  The  focus  areas  provided  a  solid  baseline  against 
which  I  could  perform  a  comparative  analysis  with  current  DoD  policy  and  guidance  and 
examine  the  extent  to  which  Agile  is  (or  isn’t)  complementary. 

More  specifically,  this  study  addresses  the  following  question:  does  the  Agile  software 
development  methodology  complement  the  DoD  acquisition  process? 
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It  is  hypothesized  that  the  DoD  acquisition  process  impedes  Agile  adoption.  While 
noticeable  effort  has  been  made  in  the  DoD  to  implement  Agile  practices,  additional  effort  must 
be  made  to  reform  the  laws,  regulations,  and  policies  that  guide  the  acquisition  process  to  enable 
an  Agile  approach. 

Based  on  previous  studies  completed  by  the  GAO  and  other  supporting  organizations,  I 
anticipated  findings  that  would  reveal  complementary  aspects  of  the  Agile  and  DoD  frameworks, 
as  well  as  areas  that  still  require  refinement  in  terms  of  DoD  implementation.  My  research  is 
intended  to  highlight  potential  aspects  within  the  DoD  acquisition  framework  that  require 
modification  to  eliminate  barriers  in  achieving  DoD  Agile  adoption. 

Limitations 

The  scope  of  this  research  was  limited  to  available  literature  from  government  sources 
and  industry  partners  who  have  adopted  Agile  IT  software  development  in  the  DoD  environment, 
or  have  assessed  the  practicability  of  its  adoption.  This  includes  technical  reports,  news  articles, 
and  presentations,  in  addition  to  applicable  DoD  laws,  regulations,  policies,  and  guidance. 

Confining  the  scope  to  the  context  of  DoD  may  have  introduced  some  limitations.  For 
example,  it  eliminated  numerous  analyses  conducted  within  the  commercial  sector  on  the 
effectiveness  of  Agile  software  development  in  non-DoD  organizations.  Given  more  time,  the 
study  could  have  included  a  general  Agile  software  development  assessment  and  leveraged 
findings  and  best  practices  from  commercial  organizations  with  considerably  more  Agile 
experience  than  DoD. 

The  primary  assumption  in  conducting  this  research  was  that  the  DoD  acquisition  process 
is  the  main  barrier  to  increased  Agile  adoption  in  the  DoD.  There  may,  however,  be  other 
conditions  that  hinder  proper  implementation  of  Agile  software  development. 
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Chapter  2  -  Literature  Review 


This  chapter  provides  a  review  of  DoD  laws,  regulations,  policies,  and  guidance;  and 
briefings,  articles,  and  technical  reports  completed  by  government  organizations  and  industry 
partners  on  the  adoptability  of  Agile  practices  within  the  DoD  environment.  The  literature  review 
analyzed  available  documentation  and  research  to  identify  commonalities,  differences,  and  gaps 
that  may  highlight  current  roadblocks  within  the  DoD  acquisition  process  that  impede  Agile 
adoption. 

Laws,  Regulations,  Policy,  and  Guidance 

1.  Better  Buying  Power  (BBP)  3.0  (Under  Secretary  of  Defense  for  Acquisition, 
Technology,  and  Logistics,  2015).  The  BBP  initiative  originated  in  2010  to  address  several 
persistent  challenges  across  the  DoD.  These  included  lack  of  affordability,  competition, 
uncontrollable  costs,  and  significant  bureaucracy  that  reduced  productivity.  Several  of  the  BBP 
3.0  focus  areas  align  to  the  Agile  software  development  methodology,  such  as  achieving 
affordable  programs,  increasing  innovation,  and  eliminating  unproductive  processes  and 
bureaucracy. 

2.  Department  of  Defense  Directive  (DoDD)  5000.01:  The  Defense  Acquisition 
System  (DoD,  2007).  DoDD  5000.01  highlights  five  policies  governing  the  Defense  Acquisition 
System:  flexibility,  responsiveness,  innovation,  discipline,  and  streamlined  and  effective 
management.  At  first  glance,  these  policies  seem  similar  to  Agile  concepts,  which  emphasize 
efficiency  and  flexibility  in  adapting  and  responding  to  stakeholder  needs.  However,  this 
directive  has  not  been  updated  since  2007.  Additional  analysis  was  required  to  determine 
whether  this  policy  document  has  kept  pace  with  increased  emphasis  on  agility  in  the  DoD. 
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3.  DoDI  5000.02:  Operation  of  the  Defense  Acquisition  System  (DoD,  2017).  DoDI 

5000.02  describes  in  detail  the  processes  and  procedures  that  drive  the  business  of  the  Defense 
Acquisition  System.  This  version  of  the  instruction  encourages  milestone  decision  authorities  to 
tailor  regulatory  requirements  and  procedures  as  needed  to  achieve  programmatic  objectives.  The 
instruction  incorporates  many  of  the  tenets  and  policies  of  the  collective  BBP  initiatives  (Kobren, 
2015),  which  may  better  facilitate  adoption  of  Agile  software  development  in  the  DoD.  Several 
models  are  provided  for  software-dominant  and  software-intensive  programs  that  would  benefit 
from  an  incremental  development  approach,  similar  to  the  Agile  concept. 

4.  DoDI  8510.01:  Risk  Management  Framework  (RMF)  for  DoD  Information 
Technology  (IT)  (DoD,  2016).  This  directive  provides  an  information  security  framework, 
including  cybersecurity,  for  the  DoD.  DoDI  8510.01  replaces  the  DoD’s  Certification  and 
Accreditation  Process,  commonly  known  as  DIACAP.  This  source  was  used  to  evaluate  findings 
identified  in  other  sources  regarding  the  compatibility  of  the  information  security  framework 
with  Agile  practices. 

5.  Federal  Acquisition  Regulation  (DoD,  2005a).  The  Federal  Acquisition  Regulation 
(FAR)  is  the  governing  document  for  the  DoD  acquisition  process  and  is  used  by  all  federal 
agencies  to  acquire  services  and  supplies.  One  source  states  that,  while  the  FAR  does  not 
explicitly  preclude  the  use  of  Agile,  it  may  present  significant  impediments  to  collaborative 
methods  such  as  Agile  software  development  (Lapham  et  al.,  2010).  The  FAR  was  used  to 
further  examine  the  area  of  contracting,  a  category  that  was  consistently  highlighted  in  many  of 
the  sources  as  a  potential  challenge  for  Agile  adoption. 


Government  Perspective 

6.  “Top-Down  IT  Approach  Too  Slow  to  Meet  Threats”  (Schoeni,  2015).  This  article 
was  written  by  an  Air  Force  (USAF)  officer  serving  the  Judge  Advocate  General  Corps. 
Schoeni’s  article  discussed  the  challenges  associated  with  cyber  warfare  and  the  snail’s  pace  at 
which  the  DoD  acquires  solutions  to  combat  it.  Schoeni  declared  there  is  an  “arsenal  of 
democracy”  (para.  1)  combatting  the  implementation  of  Agile  approaches.  This  article  was 
selected  because  it  was  one  of  only  three  sources  found  addressing  cyber  and  the  challenges 
associated  with  the  acquisition  process  that  constrain  the  use  of  Agile  concepts.  However,  while 
the  article  highlighted  a  need  to  balance  speed  with  security,  the  premise  of  this  article  was  not 
that  cyber  policy  is  an  inhibitor  of  Agile.  Rather,  Agile  was  suggested  as  a  necessary  enabler  for 
securing  solutions  to  cyber  threats  in  a  more  timely  manner.  Therefore,  although  the  article 
supports  this  study’s  hypothesis,  it  was  not  further  analyzed  for  inclusion. 

7.  “Report  of  the  Defense  Science  Board  Task  Force  on  Department  of  Defense 
Policies  and  Procedures  for  the  Acquisition  of  Information  Technology”  (DSB,  2009).  In 
this  report,  the  main  conclusion  provided  by  the  DSB  task  force  was  that  “the  conventional  DOD 
acquisition  process  is  too  long  and  too  cumbersome  to  fit  the  needs  of  the  many  IT  systems  that 
require  continuous  changes  and  upgrades”  (p.  vi).  The  task  force  provided  several 
recommendations  to  improve  the  DoD’s  IT  approach,  including  the  creation  of  a  new  acquisition 
process  specifically  for  IT.  While  the  report  was  useful  in  understanding  the  constraints  and 
challenges  that  existed  in  2009,  its  age  limited  its  applicability  for  today’s  environment.  For 
example,  a  primary  recommendation  to  provide  processes  to  accommodate  incremental 
development  approaches  was  addressed  in  the  2015  update  to  DoDI  5000.02.  For  this  reason, 
this  source  was  not  further  analyzed  for  inclusion. 
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8.  “Air  Force  Puts  22  Companies  on  $490M  Agile  Acquisition  Contract”  (McCaney, 
2015).  This  article  summarized  the  USAF’s  effort  to  put  in  place  an  indefinite- 
delivery/indefinite-quantity  contract,  as  part  of  its  initiative  to  inject  a  greater  level  of  agility  in 
the  USAF’s  acquisition  process.  Contracting  was  highlighted  in  other  sources  as  a  potential 
barrier  to  realizing  an  Agile  construct,  but  this  article  served  to  debunk  possible  myths  about 
aspects  of  the  contracting  process  that  may  be  perceived  as  inhibitors.  Additionally,  the  article 
identified  potential  lessons  learned  for  other  DoD  organizations  that  are  struggling  with  Agile  in 
the  context  of  contracting. 

9.  “DoD  CIO’s  10-Point  Plan  for  IT  Modernization”  (Takai,  2012).  In  2012,  DoD 
Chief  Information  Officer  (CIO)  Teri  Takai  shared  a  10-point  information  technology  (IT) 
modernization  plan.  Takai  provided  an  overview  of  the  state  of  the  current  DoD  IT  environment 
as  a  means  to  support  her  assertion  that  IT  modernization  is  necessary.  Some  of  the  areas 
emphasized  included  increased  demands  for  technology  and  the  sluggishness  of  DoD  IT 
programs  in  delivering  capabilities  to  the  warfighter.  Point  Four  of  the  plan  was  to  enable  Agile 
IT  through  active  user  involvement  and  more  frequent  delivery  of  useful  capabilities  (Takai, 
2012).  This  presentation  presented  Agile  as  only  one  component  in  a  larger  effort  to  streamline 
current  processes,  providing  other  aspects  to  consider  in  evaluating  whether  the  DoD  acquisition 
process  is  the  primary  obstacle  to  employing  Agile  concepts. 

10.  “Thinking  About  Agile  in  DoD”  (Welby,  2013).  This  presentation  by  the  Deputy 
Assistant  Secretary  of  Defense  for  Systems  Engineering  was  made  at  the  2013  summit,  Agile  for 
Government,  held  by  the  Association  for  Enterprise  Information  (AFEI).  In  the  brief,  Welby 
described  software  acquisition  as  “some  of  our  toughest  systems  engineering  challenges”  (p.  4) 
for  major  defense  acquisition  programs  (MDAP).  While  the  basis  of  his  assertion  was  MDAPs, 
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the  focus  of  the  presentation  was  software.  Welby  gave  his  viewpoint  on  the  challenges  posed  by 
software  acquisition,  which  include  statutory  and  regulatory  requirements.  This  perspective  is 
consistent  with  the  hypothesis  of  this  study,  and  it  provided  additional  insights  into  other  areas  of 
the  acquisition  process  that  may  obstruct  Agile  methods.  However,  Welby’s  views  may  prove  to 
be  one-sided.  The  glaring  bias  in  this  presentation  is  that  it  was  developed  for  the  AFEI  summit, 
the  theme  of  which  was  to  promote  further  adoption  of  the  Agile  methodology. 

11.  “DoD  Looks  to  Agile  to  Solve  Software  Conundrum”  (Lyngaas,  2015).  This 
article  discussed  the  DoD’s  efforts  to  implement  Agile  software  development  to  enable 
adaptability  to  changing  requirements  and  more  rapid  delivery  of  capabilities  to  the  warfighter. 
Lyngaas  quoted  William  LaPlante,  the  Assistant  Secretary  of  the  Air  Force  for  Acquisition,  who 
stated,  “There’s  a  lot  of  promise  in  agile;  I  think  a  lot  of  us  are  excited  about  it.  The  question  is 
going  to  be  how... to  take  advantage  of  it”  (para.  2).  Similar  to  Schoeni  (2015),  it  specified 
challenges  with  cybersecurity  caused  by  the  sluggishness  of  the  traditional  acquisition  process. 
However,  it  did  not  indicate  compatibility  issues  between  cybersecurity  and  Agile  practices. 
Rather,  Agile  was  seen  as  an  enabler  for  rapidly  securing  the  necessary  tools  to  counter  the  cyber 
threat.  While  this  source  presented  an  interesting  hypothesis,  it  deviates  from  the  focus  of  this 
study,  and  was  not  investigated  further. 

12.  “Software  Development:  Effective  Practices  and  Federal  Challenges  in  Applying 
Agile  Methods”  (Government  Accountability  Office  [GAO],  2012).  This  report  summarizes 
the  GAO’s  assessment  of  Agile  software  development  practices  as  a  means  to  address  many  of 
the  challenges  facing  IT  programs.  The  report  identified  14  potential  roadblocks  within  the  DoD 
related  to  adoption  of  Agile  practices.  Essentially,  most  of  the  challenges  concerned  team 
dynamics  (collaboration  and  self-directed  work,  level  of  organizational  commitment),  customer 
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buy-in,  requirements  management,  the  inflexibility  of  federal  reporting  and  documentation 
requirements,  and  an  unclear  understanding  of  Agile  within  the  DoD. 

Although  the  report  identified  many  challenges  in  adopting  Agile,  it  also  provided  best 
practices  to  implement  Agile  software  development  properly  in  the  DoD/federal  environment. 
GAO’s  report  concluded  with  the  recommendation  for  the  Federal  CIO  Council  to  include  Agile 
practices  in  its  efforts  to  promote  modular  software  development.  This  source  supports  the  null 
hypothesis  that,  while  the  adoption  of  Agile  in  the  DoD  may  pose  challenges,  there  are  no 
conditions  that  specifically  preclude  its  implementation. 

13.  “DoD  Acquisitions  Reform:  Embracing  and  Implementing  Agile”  (Oar  et  al., 
2015).  The  scope  of  this  white  paper  was  limited  to  IT  acquisition  systems,  and  it  asserted  that 
the  DoD  must  overhaul  the  acquisition  process  to  enable  execution  of  Agile  principles. 
Additionally,  the  paper  examined  areas — such  as  business  process  re-engineering,  training  (or 
lack  thereof),  and  contracting — that  inhibit  successful  Agile  implementation.  The  paper  was 
consistent  with  this  study’s  hypothesis  and  was  helpful  in  providing  the  Air  Force’s  perspective 
on  changes  that  are  needed  to  achieve  success  in  the  acquisition  process. 

Industry  Perspective 

MITRE  Corporation.  MITRE  is  a  private,  not-for-profit  organization.  The  corporation 
operates  federally  funded  research  and  development  centers  that  provide  technical  support  to 
many  DoD  and  other  government  organizations  (  “MITRE  Corporation,”  n.d.). 

14.  “How  Agile  Development  Can  Transform  Defense  IT  Acquisition”  (Messina, 
Modigliani,  &  Chang,  2015).  This  briefing  began  with  the  declaration  that  “Defense  acquisition 
processes  do  not  match  the  speed  of  new  IT  capabilities”  (p.  2).  The  declaration  is  similar  to  this 
study’s  hypothesis  that  the  current  DoD  acquisition  process  is  not  flexible  enough  to  enable 
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proper  adoption  of  Agile  software  development  across  the  DoD.  However,  the  notable  difference 
is  flexibility  versus  speed.  For  the  purposes  of  this  study,  flexibility  is  considered  to  be  an 
enabler  of  speed,  among  other  things  (e.g.  more  frequent  delivery  of  capability,  potential 
reduction  of  paperwork  and  reviews,  cost  savings,  etc.).  The  presentation  summarized  barriers  to 
Agile  adoption  that  may  still  exist  in  the  DoD  environment.  This  provided  benefit  in 
decomposing  all  of  the  sources  into  specific  focus  areas  for  the  comparative  analysis. 

15.  “Defense  Agile  Acquisition  Guide:  Tailoring  DoD  IT  Acquisition  Program 
Structures  and  Processes  to  Rapidly  Deliver  Capabilities”  (Modigliani  &  Chang,  2014).  The 
purpose  of  this  guide  was  to  demonstrate  ways  in  which  the  DoD  acquisition  framework  could 
be  modified  to  take  advantage  of  Agile  best  practices.  The  guide’s  intent  implied  that  acquisition 
processes,  in  their  current  state,  are  not  complementary  to  Agile  methods.  However,  tailoring  is 
consistent  with  policies  such  as  DoDI  5000.02,  which  explicitly  encourages  it.  Further  analysis 
of  this  source  was  conducted  to  understand  what  areas  would  require  tailoring  for  Agile  to  be 
successfully  implemented.  For  example,  while  cultural  change  could  be  considered  a  barrier  to 
Agile  adoption,  it  is  not  necessarily  a  function  of  the  acquisition  process. 

Software  Engineering  Institute  (SEI).  SEI  has  published  several  articles  and  technical 
reports  in  support  of  DoD’ s  efforts  to  evaluate  and  implement  Agile  software  development 
across  its  various  programs.  The  literature  search  identified  several  technical  reports  related  to 
the  Agile  software  development  methodology,  most  of  which  were  completed  on  behalf  of  the 
USAF. 

16.  “Agile  Methods:  Selected  DoD  Management  and  Acquisition  Concerns” 
(Lapham  et  al.,  2011).  This  technical  note  was  a  follow-on  study  to  the  2010  Lapham  et  al. 
paper,  which  is  discussed  below.  The  note  focused  on  other  aspects  that  could  be  potential 
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barriers  for  Agile  implementation,  such  as  managing  and  contracting  for  Agile  programs,  change 
management,  performance  measurement,  and  technical  milestone  reviews.  This  information  was 
helpful  in  identifying  patterns  or  common  themes  across  each  of  the  sources,  which  were  then 
further  analyzed  to  determine  their  most  recent  validity. 

17.  “Agile  Methods  in  Air  Force  Sustainment:  Status  and  Outlook”  (Regan, 
Lapham,  Wrubel,  Beck,  &  Bandor,  2014).  This  technical  note  was  completed  in  consultation 
with  individuals  from  the  USAF,  SEI,  Boeing,  Agile  &  Lean  Education  Associates,  and  others. 
The  note  investigated  how  Agile  techniques  are  applied  on  USAF  programs  in  the  area  of 
software  sustainment.  The  paper  did  indicate  the  existence  of  potential  barriers  to  using  Agile 
methods  and  examined  how  those  in  the  USAF  sustainment  community  may  have  overcome 
them.  However,  nothing  specific  was  noted  in  this  source  related  to  DoD  acquisition  policy  that 
would  disqualify  the  use  of  Agile  for  sustainment  programs. 

18.  “Considerations  for  Using  Agile  in  DoD  Acquisition”  (Lapham  et  al.,  2010).  In 
this  technical  note,  the  authors  presented  findings  based  upon  observations  and  interviews  with 
practitioners  of  the  Agile  methodology.  They  analyzed  DoDD  5000.01  and  an  earlier  version  of 
DoDI  5000.02  to  identify  considerations  for  DoD  program  offices  before  embracing  Agile.  The 
authors  concluded  that  while  the  documents  support  Agile,  they  also  recognize  challenges  and 
constraints  in  its  use.  For  example,  the  DoD  directive  identifies  challenges  in  the  areas  of  cost, 
affordability,  and  cost  realism.  According  to  the  technical  note,  “the  policy  requires  the  program 
to  determine  the  total  cost  of  ownership  which  seems  to  be  based  on  knowing  all  requirements  at 
a  detailed  level  up  front.  Agile  does  not  necessarily  support  this  concept  well  because  all  of  the 
requirements  are  not  known  at  a  detailed  level  up  front”  (p.  23). 
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In  addition  to  DoD  acquisition  policy,  Lapham  and  her  colleagues  (2010)  examined 
several  other  areas  to  answer  the  question  of  why  Agile  was  not  used  more  prevalently  in  the 
DoD.  Conclusions  included  lack  of  Agile  expertise  within  DoD,  organizational  culture  (the  DoD 
has  historically  used  a  traditional,  waterfall  approach  for  acquiring  systems  and  services),  and 
oversight  requirements  that  are  incompatible  with  Agile  (e.g.,  earned  value).  Other  areas  of 
concern  included  stakeholder  management,  integration  and  testing  (traditional  DoD  approach  is 
not  consistent  with  Agile),  and  lack  of  an  effective  program  structure  to  support  Agile 
implementation.  This  technical  note  was  helpful  in  identifying  other  potential  issues  that  may 
present  greater  challenges  than  the  DoD  acquisition  process. 

19.  “Contracting  for  Agile  Software  Development  in  the  Department  of  Defense:  An 
Introduction”  (Wrubel  &  Gross,  2015).  This  technical  note  discussed  DoD  contracting  in  the 
context  of  Agile  software  development  and  addressed  concerns  that  have  been  expressed 
regarding  the  inflexibility  of  the  contracting  process  when  Agile  development  is  desired.  It  was 
intended  to  assist  those  in  the  DoD  contracting  community,  specifically  contracting  officer 
representatives,  in  understanding  Agile  concepts.  This  source  promoted  the  use  of  Agile  software 
development  in  the  DoD  and  provided  insight  into  how  certain  contracting  processes,  policies,  or 
the  Federal  Acquisition  Regulation  could  support  the  implementation  of  Agile  methods. 

20.  “DoD  Information  Assurance  and  Agile:  Challenges  and  Recommendations 
Gathered  Through  Interviews  with  Agile  Program  Managers  and  DoD  Accreditation 
Reviewers”  (Bellomo  &  Woody,  2012).  In  this  technical  note,  Bellomo  and  Woody  examined 
the  DoD’s  Information  Assurance  Certification  and  Accreditation  Process  in  the  context  of  Agile 
development.  They  concluded  that  “while  DoD  certification  and  accreditation  processes  don’t 
prohibit  the  use  of  Agile,  use  of  them  does  introduce  some  challenges  related  to  delivering 
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software  features  rapidly  and/or  incrementally”  (p.  4).  Since  this  technical  note  was  published, 
the  DoD  has  transitioned  from  the  DIACAP  to  the  Risk  Management  Framework  (DoD,  2016). 
Further  investigation  was  conducted  to  determine  whether  the  challenges  postulated  in  this 
source  are  still  valid  today. 

21/  “Potential  Use  of  Agile  Methods  in  Selected  DoD  Acquisitions:  Requirements 
Development  and  Management”  (Nidiffer,  Garcia-Miller,  &  Carney,  2014).  This  technical 
report  described  a  qualitative  investigation  of  the  requirements  process,  including  requirements 
generation  and  management.  Because  the  report  was  limited  to  requirements  only,  it  was  not 
illuminating,  but  it  added  detail  about  how  the  acquisition  requirements  process  may 
complement  or  contradict  current  processes. 

Summary 

The  selected  references  offered  insight,  from  differing  viewpoints,  into  the  state  of  the 
DoD  acquisition  process,  as  well  as  potential  areas  within  the  acquisition  framework  that  may 
provide  challenges  for  Agile  adopters.  Some  of  the  references  were  notably  older  than  others, 
and  some  presented  one-sided  views  of  the  use  of  Agile,  having  been  designed  specifically  to 
promote  Agile  adoption  in  the  DoD.  Others  provided  a  broader  perspective  on  the  pros  and  cons 
of  Agile  software  development.  Further  examination  of  all  sources  revealed  common  themes 
used  to  examine  the  extent  to  which  Agile  is  (or  isn’t)  complementary  based  on  current  DoD 
laws,  regulations,  policy,  and  guidance. 
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Chapter  3  -  Research  Methodology 


This  chapter  defines  the  research  methodology  and  processes  used  in  the  study,  in  an 
effort  to  determine  the  degree  to  which  the  DoD  and  Agile  frameworks  are  complementary. 
Specifically,  this  paper  conducts  a  qualitative  assessment  of  previous  research  findings  to 
identify  commonalities,  differences,  and  gaps,  in  order  to  find  focus  areas  to  evaluate  against 
current  DoD  acquisition  processes. 

Research  Hypothesis 

For  this  research  project,  the  null  hypothesis  is  that  acquisition  processes  are  not  an 
inhibitor  for  adoption  of  agile  development.  The  alternative  hypothesis  is  that  the  DoD 
acquisition  process  impedes  Agile  adoption.  While  noticeable  effort  has  been  made  in  the  DoD 
to  implement  Agile  practices,  additional  effort  must  be  made  to  reform  the  laws,  regulations,  and 
policies  that  guide  the  acquisition  process  to  enable  an  Agile  approach. 

Research  Design 

The  approach  for  this  study  was  to  perform  a  qualitative  investigation  through  a 
combination  of  causal-comparative  and  historical  research.  Historical  information  was  collected 
from  publicly  available  resources,  retrieved  from  online  data  repositories.  These  sources  were 
further  analyzed  against  current  DoD  acquisition  processes  to  determine  the  extent  to  which 
Agile  is  (or  isn’t)  complementary. 

The  design  selected  for  this  study  was  two-fold.  The  first  phase  included  a  historical 
review  of  multiple  sources,  from  varying  perspectives,  focused  on  Agile  IT  software 
development  in  the  DoD  environment.  The  review  revealed  similarities  and  differences, 
shedding  light  on  potential  gaps  to  explain  why  Agile  adoption  is  still  an  issue  in  today’s 
environment.  This  information  was  then  be  used  to  develop  specific  focus  areas,  establishing  a 
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benchmark  for  the  second  phase.  Preliminary  analysis  revealed  the  following  areas,  at  minimum, 
that  were  considered  for  the  second  phase:  acquisition  oversight  (including  performance 
measurement,  reviews,  and  documentation),  requirements  management,  and  contracting. 

The  second  phase  included  a  comparative  analysis  of  the  specified  focus  areas  against 
applicable  DoD  laws,  regulations,  policies,  and  guidance.  The  goal  of  this  phase  was  to 
determine  the  degree  to  which  current  acquisition  processes  may  hinder  Agile  adoption  within 
these  areas. 

The  design  of  this  study  was  limited  by  the  nature  of  the  selected  research  strategies, 
which  may  have  introduced  some  error  in  the  findings.  All  of  the  data  used  was  static  (i.e., 
historical)  and  did  not  take  advantage  of  the  dynamic  nature  of  continued  research  in  this  area. 
Since  this  study  began,  government  organizations  and  industry  partners  continue  to  assess  ways 
in  which  the  DoD  acquisition  process  can  evolve  to  accommodate  further  adoption  of  the  Agile 
methodology.  Therefore,  the  body  of  knowledge  continues  to  grow,  making  it  impossible  to 
include  all  perspectives  and  progress  in  this  analysis.  Consequently,  it  is  conceivable  that  the 
selected  literature  did  not  include  one  or  more  of  the  issues  surrounding  Agile  adoption  in 
today’s  DoD  environment. 

Bias  and  Error 

The  researcher  of  this  study  previously  supported  a  non-ACAT  (acquisition  category)  IT 
program  within  the  U.S.  Army,  which  has  implemented  the  Agile  software  development 
methodology  to  develop  and  sustain  its  collection  of  IT  business  solutions.  This  researcher  has 
no  experience  with  implementing  Agile  methods  for  an  ACAT  program.  However,  she 
understands  some  of  the  general  challenges  associated  with  transitioning  from  a  traditional 
development  approach  (e.g.,  waterfall)  to  an  Agile  construct.  This  transition  requires  significant 
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flexibility  in  areas  such  as  stakeholder  collaboration,  leadership  buy-in  of  Agile  principles,  and 
requirements  management.  Because  of  this  author’s  personal  experience,  biases  may  have  been 
introduced  regarding  the  benefits  of  Agile  adoption  and  the  perceived  level  of  flexibility  of  the 
DoD  acquisition  process  in  terms  of  achieving  Agile  objectives. 

Additionally,  because  data  used  in  this  study  was  historical  in  nature,  there  was  no 
opportunity  to  interview  the  authors  for  clarification,  or  to  advance  this  author’s  comprehension 
of  their  findings.  As  a  result,  assumptions  were  made  regarding  the  validity  of  the  information, 
based  upon  this  author’s  interpretation  and  personal  biases  regarding  the  credibility  of  those 
sources. 

It  is  anticipated  that  this  study  will  provide  a  reasonable  framework  for  future  research. 
Specifically,  the  study  offers  a  basis  for  others  to  assess  DoD’s  progress  with  implementing 
Agile  concepts  and  the  extent  to  which  acquisition  processes  are  evolving  to  accommodate  Agile 
approaches. 
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Chapter  4  -  Findings 


The  previous  chapter  described  this  study’s  research  methodology  as  a  two-pronged 
approach  to  investigate  the  hypothesis  that  the  DoD  acquisition  process  impedes  Agile  adoption. 
This  chapter  presents  the  results  of  the  investigation.  Numerous  sources  were  reviewed  that 
identified  both  complementary  aspects  and  potential  conflicts  between  Agile  IT  software 
development  and  the  DoD  acquisition  framework.  The  conflicting  areas  were  then  further 
analyzed  to  determine  the  extent  to  which  they  may  still  obstruct  wider  adoption  of  Agile 
practices  within  the  DoD. 

Collected  Data 

A  historical  review  of  the  sources  identified  in  Chapter  2  revealed  similar  findings  in 
terms  of  potential  areas  within  the  acquisition  framework  that  may  provide  challenges  for 
inexperienced  DoD  adopters.  A  common  deduction  found  in  multiple  sources  was  that  a  primary 
barrier  to  adopting  Agile  in  the  DoD  may  be  cultural  (Lapham  et  al.,  2010).  Other  sources 
pinpointed  specific  areas,  such  as  contracting,  that  should  be  considered  while  reforming  the 
acquisition  process  (Wrubel  &  Gross,  2015).  In  their  technical  note,  Lapham  et  al.  (2010) 
presented  findings  based  on  interviews  with  practitioners  of  the  Agile  methodology.  They  also 
performed  an  assessment  of  the  DoD  5000  series  directives  and  instructions  to  provide 
considerations  for  program  offices  before  embracing  Agile.  The  authors  concluded  that  the 
documents  support  Agile,  but  also  identify  challenges  and  constraints  in  its  use  (Lapham  et  al., 
2010). 

Multiple  areas  were  emphasized  as  potential  barriers  for  implementing  Agile  practices 
within  DoD.  While  many  of  these  areas  fell  directly  within  the  realm  of  the  DoD  acquisition 
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framework,  some  of  them  did  not  (e.g.,  culture).  A  summary  of  the  identified  areas  by  source 
appears  in  Table  1. 

It  is  important  to  note  that  the  table  is  not  intended  to  demonstrate  whether  the  sources 
agree  or  disagree  with  one  another.  Many  of  the  sources  were  limited  to  discussing  specific  areas 
such  as  contracting  or  requirements.  Rather,  the  table’s  sole  purpose  is  to  identify  those 
references  that  highlighted  certain  areas  as  potential  obstacles  for  Agile  adoption. 
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Table  1  -  Identified  Conflicts  Between  Agile  and  DoD 


Literature  Review 
Source 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

Specific  to  DoD 
Acquisition 

Framework 

Acquisition  Oversight 

X 

X 

X 

X 

Contracting 

X 

X 

X 

X 

X 

X 

X 

X 

X 

Cost  Estimation 

X 

X 

X 

Information  Assurance 

X 

X 

Program  Cost  & 
Performance 

Monitoring 

X 

X 

X 

X 

Requirements 

Management 

X 

X 

X 

X 

X 

X 

Not  Specific  to  DoD 
Acquisition 

Framework 

Culture 

X 

X 

End  User  Involvement 

X 

X 

X 

X 

Infrastructure 
(establishing  / 
maintaining  technical 
environment) 

X 

X 

Management 

Commitment 

X 

X 

X 

Team  Organization 

X 

X 

X 

X 

X 

Training  /  Lack  of 
Guidance 

X 

X 

X 

X 

X 

X 

X 

X 

For  the  purposes  of  this  study,  areas  categorized  as  not  specific  to  the  DoD  acquisition 
framework  are  not  discussed  further.  While  the  DoD’s  culture  and  organizational  construct,  for 


example,  may  be  valid  obstacles  for  Agile  adoption,  they  are  not  directly  attributable  to  existing 


acquisition  policy,  law,  or  regulation. 


Particular  findings  identified  within  the  other  areas  are  discussed  in  detail  below. 


Acquisition  Oversight.  The  DoD  acquisition  framework  is  articulated  in  a  series  of 


directives  and  instructions  that  mandate  a  certain  level  of  oversight  for  acquisition  programs. 


This  includes  milestone  reviews  and  associated  documentation  that  identify  system  requirements, 
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design,  and  test  readiness  to  inform  decisions  along  the  acquisition  life  cycle.  The  GAO  (2012) 
published  a  report  pinpointing  14  challenges  that  DoD  programs  may  encounter  when  attempting 
to  apply  Agile  practices.  Among  them  were  conflicts  between  the  perceived  rigidity  of  the 
traditional  oversight  structure  and  foundational  Agile  concepts  highlighting  the  need  for 
flexibility  in  how  efforts  are  managed  and  documented.  The  report  noted  the  difficultly  of 
integrating  compliance  reviews  into  the  fairly  short  development  iterations  inherent  in  Agile 
projects.  “Compliance  reviewers  queued  requests  as  they  arose  and... the  reviews  could  take 
months  to  perform.  This  caused  delays  for  iterations  that  needed  such  reviews  within  the  few 
weeks  of  the  iteration”  (GAO,  2012,  p.  19). 

Furthermore,  other  sources  highlighted  documentation  challenges  when  aligning  Agile 
with  the  traditional  oversight  construct.  Agile  documentation  is  deliberately  kept  to  a  minimum, 
and  it  evolves  over  time  (updated  with  each  increment),  which  may  not  be  adequate  for 
milestone  reviews  such  as  a  critical  design  review  (CDR;  Lapham  et  al.,  2010). 

As  noted  in  two  of  the  sources,  implementation  of  Agile  practices  in  the  sustainment  of 
software  systems  has  proven  successful  with  respect  to  project  execution,  cost  estimation,  and 
contracting.  Sustainment  programs  are  not  subject  to  the  design  and  requirements  volatility  of 
development  programs.  Also,  contracting  vehicles  are  often  service  based,  facilitating  a  more 
accurate  estimation  of  required  support  resources  (Lapham  et  al.,  2011).  As  Lapham  and  her 
colleagues  (2011)  noted,  “the  biggest  difference  in  sustainment  is  that  an  architecture  for  the 
system  has  been  defined  and  implemented”  (p.  64). 

Analysis.  DoDD  5000.01  (DoD,  2007)  highlights  five  policies  governing  the  Defense 
Acquisition  System:  flexibility,  responsiveness,  innovation,  discipline,  and  streamlined  and 
effective  management.  The  first  three  are  integral  characteristics  of  the  Agile  methodology 
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(Lapham  et  al.,  2010).  Regarding  flexibility  and  streamlined  management,  the  directive 
encourages  program  managers  (PMs)  and  milestone  decisions  authorities  (MDAs)  to  tailor 
documentation  and  oversight  requirements  as  appropriate,  subject  to  applicable  regulations  and 
laws.  In  its  emphasis  on  the  importance  of  flexibility,  the  directive  affirms  that  “there  is  no  one 
best  way  to  structure  an  acquisition  program  to  accomplish  the  objective  of  the  Defense 
Acquisition  System.  MDAs  and  PMs  shall  tailor  program  strategies  and  oversight,  including 
documentation  of  program  information. .  .to  fit  the  particular  conditions  of  that  program. . .” 
(DoD,  2007,  p.  3). 

DoDD  5000.01  (DoD,  2007)  also  promotes  incremental  development  to  facilitate  greater 
responsiveness  in  delivering  capabilities.  Incremental  development  is  compatible  with  the 
iterative  nature  of  Agile  IT  software  development.  And  while  documentation  does  evolve 
incrementally  in  Agile  development,  proper  planning  can  be  done  beforehand  to  ensure  that 
documentation  completion  and  delivery  schedules  are  sufficient  to  meet  CDR  and  other 
milestone  review  requirements  (Lapham  et  al.,  2010). 

DoDI  5000.02  (DoD,  2017)  includes  acquisition  program  models  specifically  designed 
for  software-intensive  programs.  Model  3  accommodates  the  iterative  development  nature  of 
many  software  programs,  whereby  capabilities  are  delivered  incrementally.  “Each  deployment 
will  result  from  a  specific  build  and  provide  the  user  with  a  mature  and  tested  sub-element  of  the 
overall  incremental  capability.  Several  builds  and  deployments  will  typically  be  necessary  to 
satisfy  approved  requirements  for  an  increment  of  capability”  (p.  13).  The  instruction  states  the 
importance  of  properly  structuring  these  types  of  programs  so  they  are  not  “overwhelmed  with 
frequent  milestone  or  deployment  decision  points  and  associated  approval  reviews... multiple 
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activities  or  build  phases  may  be  approved  at  any  given  milestone  or  decision  point,  subject  to 
adequate  planning,  well-defined  exit  criteria,  and  demonstrated  progress”  (p.  14). 

DoDI  5000.02  (DoD,  2017)  also  encourages  the  tailoring  of  milestone  documentation  as 
appropriate.  Explicitly,  “documents  prepared  in  support  of  the  decision  process  (e.g.,  Acquisition 
Strategy,  Systems  Engineering  Plan  (SEP),  Test  and  Evaluation  Master  Plan  (TEMP),  Life-Cycle 
Sustainment  Plan  (LCSP))  should  generally  not  be  prepared  solely  for  staff  review  and  approval, 
but  be  intended  primarily  for  use  within  the  program  as  planning  and  management  tools  that  are 
highly  specific  to  the  program  and  tailored  to  meet  program  needs”  (p.  4). 

Contracting.  Several  sources  noted  challenges  specifically  in  the  area  of  contracting. 
These  included  staffing  and  monitoring  contractor  performance  (GAO,  2012),  developing 
requests  for  proposal  (RFPs;  Lapham  et  al.,  2010),  and  a  common  perception  within  the  DoD 
that  the  contracting  process  is  fundamentally  inflexible.  The  GAO  (2012)  noted  that  federal 
procurement  practices  do  not  enable  the  level  of  flexibility  required  to  ramp  up  resources  when 
needed  for  Agile  projects.  Challenges  are  incurred  when  “changing  contractor  staff  in  time  to 
meet  iteration  time  frames  and  that  accommodating  task  changes  from  one  iteration  to  the  next  is 
challenging  because  contracting  officers  require  cumbersome  traditional  structured  tasks  and 
performance  checks”  (p.  18). 

With  respect  to  RFP  development,  traditional  proposals  customarily  require  a  series  of 
contract  data  requirements  lists  (CDRLs),  which  document  contractor  progress/performance 
(Lapham  et  al.,  2010).  Lapham  and  her  colleagues  asserted  that  “this  level  of  documentation  is 
contrary  to  Agile  precepts  of  creating  ‘just  enough’  documentation”  (p.  13).  The  authors 
rationalized  that  it  is  feasible  to  develop  a  more  streamlined  set  of  deliverables,  while  still 
complying  with  regulatory  requirements,  with  the  proper  amount  of  coordination  and 
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collaboration  between  the  contractor  and  government.  However,  they  also  noted  that  “the  Federal 
Acquisition  Regulations  (FARs)  impose  significant  obstacles  to  collaborative  endeavors.  In  fact, 
since  the  system  tries  to  encourage  competition.. .in  many  cases  users  are  actually  prevented  from 
collaborating  with  system  developers  until  late  in  the  acquisition  life  cycle”  (p.  27). 

In  a  MITRE  report,  Modigliani  and  Chang  (2014)  asserted  that  “the  current  contracting 
environment  does  not  encourage  Agile  approaches”  (p.  33),  and  identified  specific  areas  in 
which  traditional  contracting  practices  and  Agile  do  not  align.  The  areas  included  long 
contracting  timelines,  emphasis  on  technical  solution  versus  technical  capability,  the  fact  that 
requirements  are  essentially  locked  in  at  award  (requiring  contract  modifications  if  changes  are 
needed).  Also  highlighted  were  a  less  collaborative  contractor-government  relationship  and 
centralized  (versus  embedded)  contracting  support  that  cannot  respond  rapidly  to  needed 
contracting  actions.  While  the  authors  underscored  challenges,  they  also  offered  potential 
solutions  such  as  establishing  contract  vehicles  at  the  enterprise  level,  instead  of  at  the  program 
level,  in  order  to  reduce  contracting  timelines.  “Contract  vehicles  suitable  for  Agile  development 
have  pre-established  contract  pricing,  terms  and  conditions,  and  pre-vetted  qualified  vendor(s). 
This  allows  task  orders  to  be  issued  in  a  matter  of  weeks  versus  months”  (p.  36). 

In  contrary  to  the  challenges,  other  sources  highlighted  successful  contract 
implementations  supporting  Agile  software  development  and  offered  best  practices/lessons 
learned  in  the  areas  of  contract  strategy  selection  and  oversight.  One  source  posited  that  the 
difficulties  incurred  when  implementing  Agile  contracting  practices  in  the  DoD  are 
predominantly  caused  by  lack  of  awareness/training  (Wrubel  &  Gross,  2015).  Because  Agile  is 
intrinsically  different  from  traditional  software  development,  contracting  officers  who  lack  Agile 
experience  and  training  may  have  difficulty  adjusting  to  new  methods  (Oar  et  al.,  2015). 
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Wrubel  and  Gross  (2015)  also  pointed  to  a  lack  of  Agile- related  education  opportunities 
in  the  DoD  for  contracting  officers.  Additionally,  the  authors  determined  that  “future  work  is 
needed  to  collect  more  examples  and  approaches  for  effective  contracts.  This  information  can 
help  form  a  body  of  practice  for  government  organizations  to  approach  software  development 
work,  using  the  Agile  values  and  principles”  (p.  44). 

Analysis.  Collaboration  and  flexibility  are  key  components  of  Agile  (Lapham  et  al., 
2010),  which  are  both  emphasized  in  DoDD  5000.01  (DoD,  2007).  According  to  the  directive, 
MDAs  and  PMs  are  permitted  to  tailor  program  documentation  as  needed  and  within  the  bounds 
of  relevant  laws  and  regulations.  It  is  plausible  to  assume  that  CDRLs  fall  within  this  category. 

Collaboration  and  innovation  are  encouraged  in  the  FAR  (DoD,  2005a),  the  guiding 
document  for  contracting  goods  and  services  in  the  DoD  acquisition  environment.  Subpart  1.1 
defines  an  empowered  acquisition  team  permitted  to  determine  the  most  suitable  approaches  for 
managing  their  contracts,  subject  to  limitations  of  the  FAR  and  other  applicable  laws, 
regulations,  and  policies.  The  FAR  further  states  that  “if  a  policy  or  procedure,  or  a  particular 
strategy  or  practice,  is  in  the  best  interest  of  the  Government  and  is  not  specifically  addressed  in 
the  FAR,  nor  prohibited  by  law  (statute  or  case  law),  Executive  order  or  other  regulation, 
Government  members  of  the  Team  should  not  assume  it  is  prohibited.  Rather,  absence  of 
direction  should  be  interpreted  as  permitting  the  Team  to  innovate  and  use  sound  business 
judgment  that  is  otherwise  consistent  with  law  and  within  the  limits  of  their  authority”  (p.  1.1-2). 

Cost  Estimation.  Cost  estimation  occurs  throughout  the  entire  life  of  DoD  programs, 
including  generating  a  life-cycle  cost  estimate  prior  to  Milestones  A  and  B  (Lapham  et  al.,  2010). 
“The  program  life  cycle  cost  estimate  (PLCCE)...is  presented  to  the  program’s  Milestone 
Decision  Authority  (MDA)  at  each  milestone.  The  PLCCE  must  look  forward  from  the  current 
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program  state  to  the  end  of  the  system’s  life,  and  assess  the  cost  of  the  product  or  system  over  its 
entire  life”  (p.  47). 

According  to  an  SEI  technical  report,  DoDD  5000.01  requires  that  programs  provide  total 
ownership  costs  up  front  (Lapham  et  al.,  2010),  which  presumes  that  requirements  are  well 
defined  at  the  start  of  a  program.  Additionally,  the  Program  Stability  policy  in  DoDD  5000.01 
(DoD,  2007)  states  that  programs  are  generally  not  fully  funded  until  the  system  design  has  been 
chosen.  Because  of  the  evolving  nature  of  both  the  requirements  and  design  in  an  Agile  context, 
realistically  estimating  life  cycle  costs  may  prove  challenging. 

Moreover,  some  would  argue  that  because  the  design  changes  over  time,  Agile  does  not 
support  the  Program  Stability  policy  (Lapham  et  al.,  2010).  As  a  workaround  to  the  design 
challenge,  the  authors  offered  that  Agile  does  support  an  overall  architectural  framework  that 
may  enable  policy  compliance.  However,  this  approach  would  need  to  be  coordinated  with  the 
MDA,  who  is  the  ultimate  decision  authority. 

The  GAO  (2012)  noted  that  “traditional  oversight  requires  detailed  artifacts  in  the 
beginning  of  a  project,  such  as  cost  estimates... requiring  these  artifacts  so  early  was  challenging 
because  it  was  more  worthwhile  to  start  with  a  high-level  cost  estimate  and  vision  to  be  updated 
as  the  solution  was  refined  through  iterations,  rather  than  spending  time  estimating  costs  and 
strategies  that  may  change”  (p.  20). 

MITRE’s  analysis  determined  that  “estimating  costs  in  an  Agile  environment  requires  a 
more  iterative,  integrated,  and  collaborative  approach  than  in  traditional  acquisition  programs. 
While  a  program  can  develop  rough  order  of  magnitude  estimates  in  the  beginning,  it  cannot  gain 
an  understanding  of  costs  and  schedule  with  any  true  fidelity  until  the  development  teams  are  in 
a  rhythm”  (Modigliani  &  Chang,  2014,  p.  41). 
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Analysis.  DoDD  5000.01  (DoD,  2007)  defines  several  policies  related  to  cost 
affordability  and  realism  that  may  not  entirely  align  to  the  Agile  framework.  The  Program 
Stability  policy  states  that  “DoD  Components  shall  develop  realistic  program  schedules,  long- 
range  investment  plans,  and  affordability  assessments,  and  shall  strive  to  ensure  stable  program 
funding.  The  MDA  shall  determine  the  appropriate  point  at  which  to  fully  fund  an  acquisition 
program,  generally  when  a  system  concept  and  design  have  been  selected... capability  needs 
have  been  approved,  and  system-level  development  is  ready  to  begin”  (p.  9).  In  an  Agile  context, 
the  policy  may  be  supportable,  depending  on  the  type  of  system,  the  level  of  detail  needed  to 
support  a  decision,  and  the  position  of  the  MDA  (Lapham  et  al.,  2010). 

The  Cost  and  Affordability  Policy  in  DoDD  5000.01  (DoD,  2007)  discusses  cost  as  an 
independent  variable  and  fiscal  constraints.  “To  the  greatest  extent  possible,  the  MDAs  shall 
identify  the  total  costs  of  ownership,  and  at  a  minimum,  the  major  drivers  of  total  ownership 
costs.  The  user  shall  address  affordability  in  establishing  capability  needs”  (p.  5).  The  challenges 
of  this  policy  are  not  necessarily  unique  to  Agile  efforts,  as  many  traditionally  structured 
programs  face  affordability  challenges.  However,  not  having  fully  defined  requirements  up  front 
is  arguably  a  disadvantage.  The  constantly  evolving  requirements  mean  that  cost  estimates  are 
also  continually  refined  over  time,  which  introduces  a  degree  of  instability  in  terms  of  managing 
cost  and  affordability. 

Information  Assurance.  Information  assurance  (IA)  is  defined  broadly  as  “the  technical 
and  managerial  measures  designed  to  ensure  the  confidentiality,  possession  or  control,  integrity, 
authenticity,  availability  and  utility  of  information  and  information  systems”  (“Information 
Assurance,”  n.d.).  Cybersecurity  is  considered  a  subset  of  IA,  and  is  defined  in  DoDI  8510.01 
(DoD,  2016)  as  the  “prevention  of  damage  to,  protection  of,  and  restoration  of  computers, 
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electronic  communications  systems,  electronic  communications  services,  wire  communication, 
and  electronic  communication,  including  information  contained  therein,  to  ensure  its  availability, 
integrity,  authentication,  confidentiality,  and  nonrepudiation”  (p.  47). 

Several  sources  emphasized  the  sluggishness  of  the  DoD  acquisition  process  as  a  primary 
challenge  in  delivering  capabilities  when  needed.  Specific  areas  noted  were  acquisition 
oversight,  contracting,  and  the  IA  certification  and  accreditation  process.  Regarding  IA,  software 
continues  to  be  an  integral  part  of  the  DoD  capability  portfolio.  As  such,  the  pace  of  the 
acquisition  process  may  not  be  able  to  keep  up  with  the  speed  at  which  new  information-security 
threats  emerge  (Modigliani  &  Chang,  2014). 

In  their  assessment  of  the  DIACAP  against  the  Agile  framework,  Bellomo  and  Woody 
(2012)  noted  that  while  nothing  within  the  IA  process  technically  precludes  incremental  software 
development,  “there  are  aspects  of  DIACAP  that  make  incremental  delivery  of  security 
requirements  challenging”  (p.  5).  According  to  their  SEI  report,  a  source  of  this  challenge  is  the 
IA  review  process.  For  example,  while  DIACAP  does  allow  for  iterative  development,  the  series 
of  review/approval  gates  inherent  in  the  accreditation  process  are  suited  for  a  traditional 
project/program  life  cycle  (i.e.,  waterfall).  In  the  traditional  model,  security  requirements  are  not 
evaluated  until  development  is  complete. 

Bellomo  and  Woody  (2012)  also  described  a  general  perspective  that  “the  information 
assurance  accreditation  delay  is  so  extensive  (often  months  to  a  year)  that  the  DIACAP  process 
almost  negates  the  benefits  gained  through  rapid  development  methods”  (p.  5).  Those  who  were 
interviewed  for  their  study  identified  the  following  challenges  associated  with  DIACAP: 

•  The  lengthy  accreditation  testing  phase  causes  delays  in  releasing  capability. 

•  Security  requirements  must  be  prioritized  differently  due  to  risk. 
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•  There  is  a  lack  of  security  engineering  expertise  in  the  DoD. 

•  The  intent/purpose  of  many  of  the  IA  controls  is  not  well  understood. 

•  The  reaccreditation  process  for  iterative  releases  is  unclear  and  confusing. 

•  Use  of  test  case-driven  verification  approaches  for  IA  validation  is  limited. 

•  Documentation  requirements  exceed  what  the  Agile  process  normally  produces  (e.g., 
architecture  views,  security  design  documentation). 

•  Collaboration  with  accrediting  authorities  can  be  challenging. 

•  There  is  no  incentive  to  develop  security  features  in  early  Agile  increments. 

•  DIACAP  includes  traditional  milestone  review  artifacts  that  don’t  align  to  Agile. 

•  The  joint  services  accreditation  process  is  challenging 

In  addition  to  identifying  challenges  associated  with  DIACAP,  Bellomo  and  Woody 
(2012)  summarized  recommendations  to  mitigate  potential  delays  and  address  the  foregoing 
issues.  This  included  defining  reaccreditation  criteria  up  front  and  early,  greater 
collaboration/coordination  between  the  IA  and  Agile  communities,  and  integrating  security  and 
Agile  best  practices. 

Analysis.  The  2017  update  of  DoDI  5000.02  includes  an  enclosure  dedicated  to 
cybersecurity.  The  new  enclosure  communicates  the  importance  of  incorporating  cybersecurity 
across  the  entire  life  cycle  of  acquisition  programs  (DoD,  2017).  The  instruction  also  focuses  on 
cybersecurity  test  and  evaluation  and  on  ensuring  a  strategy  is  fully  addressed  in  the  Test  and 
Evaluation  Master  Plan.  As  a  means  to  mitigate  cybersecurity  risks,  the  instruction  provides 
guidance  on  designing  for  the  cyber  threat  environment.  Among  others,  PMs  are  directed  to 
“ensure  cybersecurity  and  related  system  security  requirements,  design  characteristics,  and 
verification  methods  to  demonstrate  the  achievement  of  those  requirements  are  included  in  the 
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technical  baseline  and  maintain  bi-directional  traceability  among  requirements  throughout  the 
system  life  cycle”  (p.  174).  As  mentioned  earlier,  some  Agile  projects  do  not  incorporate  security 
features  until  later  iterations.  However,  this  can  most  likely  be  adjusted  to  accommodate  DoD 
guidance. 

The  RMF  (DoDI  8510.01)  mandates  that  the  DoD  “establish  and  use  an  integrated 
enterprise-wide  decision  structure  for  cybersecurity  risk  management”  (DoD,  2016,  p.  2)  and 
comply  with  the  governance  process  defined  within  the  instruction.  While  it  is  fairly  prescriptive 
in  defining  the  required  security  controls,  documentation,  and  reviews,  it  does  make  allowances 
for  tailoring  in  some  cases.  For  example,  DoD  organizations  are  allowed  to  tailor  the  set  of 
security  controls  as  appropriate  when  developing  RFPs  for  IT  services  (DoD,  2016).  This 
researcher  could  find  nothing  in  this  DoD  instruction,  or  the  other  sources  mentioned  above,  that 
would  specifically  disqualify  the  use  of  Agile  IT  software  development. 

Program  Cost  and  Performance  Monitoring.  For  the  purposes  of  this  study,  program 
cost  and  performance  monitoring  include  the  mechanisms  used  by  program  offices  to  monitor 
contractor  performance  in  executing  the  technical  requirements  of  its  contract.  Specifically, 
programs  use  integrated  master  schedules  (IMS)  and  contractor  earned  value  management 
(EVM)  systems  to  monitor  progress.  ACAT  programs  leverage  EVM  data  to  assess  contractor 
performance  against  the  contractor’s  cost  estimate  (Lapham  et  al.,  2011).  The  IMS  is  an 
integrated  schedule  consisting  of  all  of  the  work  activities  (tasks,  events,  and  accomplishments) 
used  to  manage  an  acquisition  program  on  a  daily  basis  (DoD,  2005b). 

The  dynamic  nature  of  the  Agile  requirements  and  design  processes  requires  a  shift  from 
the  customary  IMS  construct.  Schedules  developed  for  non- Agile  (waterfall)  projects  include 
detailed  and  relatively  fixed  work  breakdown  structures  based  upon  specific  phases  and  project 
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deliverables  (e.g.,  requirements,  design,  development,  test,  etc.).  In  Agile  projects,  requirements 
constantly  evolve,  as  can  the  scope  of  each  iteration.  This  level  of  fluidity  can  complicate  a 
program’s  ability  to  track  performance  via  traditional  EVM  and  to  maintain  an  accurate  IMS 
(Lapham  et  al.,  2010). 

The  GAO  (2012)  underscored  a  misalignment  between  traditional  contractor  status 
tracking  and  Agile  practices.  While  the  challenge  associated  with  project  status  tracking  was  due 
to  lack  of  experience,  use  of  EVM  was  hampered  by  lack  of  guidance  for  projects  of  an  iterative 
nature.  One  DoD  official  communicated  that  “changes  were  viewed  as  control  problems  rather 
than  as  revisions  to  be  expected  during  an  iteration”  (p.  20). 

In  their  assessment  on  the  value  of  EVM  in  the  Agile  context,  Modigliani  and  Chang 
(2014)  concluded  that  “given  the  dynamic  and  iterative  structure  and  processes  of  Agile, 
implementing  an  EVM  system  can  pose  a  significant  challenge  with  little  value”  (p.  48). 

In  SEI’s  technical  note,  the  authors  declared  that  EVM  can  be  adapted  for  use  on  Agile 
programs.  However,  the  process  would  require  a  significant  amount  of  coordination  between  the 
government  and  contractor  teams,  and  it  could  be  labor  intensive  (Lapham  et  al.,  2010). 

Analysis.  DoDI  5000.02  (DoD,  2017)  provides  guidelines  regarding  EVM  applicability 
for  programs.  EVM  applicability  is  determined  by  contract  type,  duration,  and  value.  It  is 
required  for  programs  with  a  contract  value  greater  than  $20  million,  and  applies  “to  cost 
reimbursable  or  incentive  contracts,  inclusive  of  options,  with  18  months  or  greater  period  of 
performance  and  based  on  the  nature  of  the  work  scope”  (p.  79).  According  to  the  instruction, 
EVM  is  considered  “one  of  DoD’s  and  industry’s  most  powerful  program  planning  and 
management  tools... normally  used  in  conjunction  with  cost  plus  and  fixed-price  incentive 
contracts  with  discrete  work  scope”  (p.  89). 
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Based  on  this  guidance,  not  all  contracts/programs  are  required  to  use  EVM.  For 
example,  EVM  does  not  apply  to  services  contracts.  “Services  contracts  are  commonly  used  in 
the  commercial  sector  for  Agile  development  because  of  its  flexibility,  although  they  are  not 
typically  preferred  in  the  DoD  environment  because  the  contractor  is  not  necessarily  incentivized 
to  control  costs”  (Modigliani  &  Chang,  2014,  p.  38).  In  addition,  not  all  contract  vehicles  are 
suitable  for  Agile  development  programs.  Modigliani  and  Chang  (2014)  noted  that  “the  Agile 
development  process  is  characterized  by  constant  change  and  reprioritization  of  requirements. 
This  makes  it  impractical  to  select  an  Agile  development  contractor  using  a  contract  type  that 
locks-in  requirements  up  front  and  defines  end-state  products  on  a  completion-basis”  (p.  35).  The 
viability  of  EVM  for  Agile  programs  will  depend  on  securing  a  suitable  contract  type  that 
accommodates  an  Agile  approach. 

Requirements  Management.  The  fluctuating  nature  of  the  requirements  in  an  Agile 
framework  has  been  discussed  in  many  sections  of  this  research  paper.  As  such,  it  can  be 
considered  a  broad  area  that  affects  other  areas  such  as  culture,  contracting,  and  acquisition 
oversight.  Not  only  can  new  requirements  be  identified  as  efforts  progress,  but  requirements  can 
move  from  one  iteration  to  the  next,  or  be  deferred  to  subsequent  increments/releases.  This  level 
of  volatility  can  create  challenges  in  managing  these  requirements. 

SEI  summarized  the  most  significant  issues  identified  in  their  investigation  of 
implementing  an  Agile  requirements  methodology  in  the  DoD:  lack  of  “accepted  DoD-level 
guidance  for  tracking  progress,  translating  potential  Agile  measures  of  progress  to  earned  value 
measures,  a  risk-averse  acquisition  culture,  work  breakdown  structure  as  a  tool  to  manage  task 
assignments,  effect  of  requirements  change  on  contracts,  and  perception  that  reduced 
documentation  is  a  cause  for  concern”  (Nidiffer  et  al.,  2014,  p.  18). 
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The  GAO  (2012)  identified  requirements  management  as  one  of  14  challenges  in 
implementing  Agile  in  the  DoD.  “Teams  had  difficulty  managing  iterative  requirements. .  .one 
official  reported  that  customers  were  initially  challenged  to  validate  and  prioritize  which 
requirements  would  be  assigned  to  a  release... The  second  official  said  they  were  challenged  to 
accommodate  new  requirements  within  the  fixed  schedule  for  a  product  release”  (GAO,  2012,  p. 
19). 

Some  sources  emphasized  the  shortcomings  of  the  acquisition  process  in  accommodating 
requirements  changes.  Any  changes  may  trigger  “a  very  rigorous  engineering  change  proposal 
process.  The  sheer  amount  of  documentation  required  in  this  process  causes  schedule  delays  and 
is  accompanied  by  a  hefty  price  tag”  (Oar  et  al.,  2015,  p.  3). 

One  advantage  of  the  level  of  requirements  flexibility  inherent  in  Agile  efforts  is  that 
capabilities  are  allowed  to  evolve  based  on  user  needs.  Welby  (2013)  compared  traditional  and 
incremental  planning  processes  for  software  development.  The  incremental  planning  approach  in 
Agile  efforts  allows  plans  to  “adapt  to  changes  without  causing  rework,  waste,  and  development 
cost  growth”  (slide  10). 

A  MITRE  report  described  the  concept  of  the  IT  Box  model  for  information  systems. 

This  model  is  part  of  the  Joint  Capabilities  Integration  &  Development  System  (JCIDS)  process 
and  is  included  in  the  JCIDS  Manual  published  by  the  Joint  Requirement  Oversight  Council. 
JCIDS  implements  the  DoD  requirements  process,  and  “supports  the  Chairman  of  the  Joint 
Chiefs  of  Staff  (CJCS)  and  the  Joint  Requirements  Oversight  Council  (JROC)  in  identifying, 
assessing,  and  prioritizing  joint  military  capability  needs  as  required  by  law”  (“Joint 
Capabilities,”  n.d.).  The  IT  Box  Model  serves  as  a  distinct  advantage  for  IS  programs:  an  Initial 
Capabilities  Document  (ICD)  replaces  the  requirement  for  both  a  Capability  Development 
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Document  (CDD)  and  Capability  Production  Document  (CPD).  The  benefit  of  the  ICD  is  that  it 
reduces  the  documentation  requirement  and  complements  the  evolving  nature  of  the 
requirements  in  Agile  projects.  “In  lieu  of  CDDs  and  CPDs,  programs  can  develop  Requirements 
Definition  Packages  (RDPs)  to  capture  a  subset  of  the  IS  ICD  scope...  the  requirements 
documents  are  designed  for  a  smaller  scope  of  work  and  approval  at  a  lower  level.  This 
flexibility  and  streamlining  of  IT  requirements  enables  Agile  development  within  a  DoD 
program”  (Modigliani  &  Chang,  2014,  p.  21). 

Analysis.  Based  on  the  results  of  the  historical  review,  some  of  the  challenges  identified 
are  not  necessarily  attributable  to  constraints  imposed  by  DoD  acquisition  law,  regulation,  or 
policy.  For  example,  the  issue  of  managing  and  prioritizing  requirements,  identified  by  the  GAO, 
appears  to  be  more  a  function  of  training  and  collaboration  than  DoD  policy. 

While  programs  are  required  to  comply  with  DoD  guidance,  JCIDS  appears  to  provide  a 
reasonable  amount  of  flexibility  for  information  system  programs  that  permit  a  more  efficient 
approach  to  requirements  management.  “As  long  as  the  program  operates  within  these  four  sides 
of  the  IT  Box,  they  need  not  return  to  the  JROC  for  approval  or  oversight”  (Modigliani  &  Chang, 
2014,  p.  21). 

Summary 

The  historical  review  revealed  six  areas  specific  to  the  DoD  acquisition  framework  that 
were  highlighted  in  at  least  one  source  as  potential  constraints  to  implementing  Agile  practices  in 
the  DoD  environment: 

•  Acquisition  oversight 

•  Contracting 

•  Cost  estimation 
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•  IA 


•  Program  cost  and  performance  monitoring 

•  Requirements  management 

These  areas  were  then  evaluated  against  available  DoD  policies,  regulations,  and  laws  to 
identify  any  conditions  that  would  inhibit  the  successful  propagation  of  Agile  concepts  in  the 
DoD. 

From  an  acquisition  oversight  perspective,  the  DoD  5000  series  directives  and 
instructions  encourage  MDAs  and  PMs  to  tailor  program  oversight  and  documentation  as 
appropriate,  in  compliance  with  applicable  law  and  policy.  DoDD  5000.01  (2007)  also  promotes 
incremental  development  to  facilitate  greater  responsiveness  in  delivering  capabilities. 
Incremental  development  is  compatible  with  the  iterative  nature  of  Agile  IT  software 
development. 

The  contracting  process  was  heavily  emphasized  as  posing  significant  challenges.  Agile 
programs  are  susceptible  to  lengthy  contracting  timelines,  and  the  administrative  burdens  of 
contract  modifications  for  any  changes  to  the  requirements. 

Based  on  the  available  data  found  within  the  sources,  cost  estimation  and  realism  early  in 
a  program  appeared  to  be  a  valid  concern.  DoD  policy  requires  that  life-cycle  cost  be  estimated 
up  front,  and  the  Program  Stability  Policy  states  that  programs  are  generally  not  fully  funded 
until  the  system  design  has  been  chosen.  These  conditions  can  prove  challenging  for  Agile 
projects,  which  are  subject  to  requirements  and  design  volatility. 

Information  assurance  is  a  broad  area  in  which  policy  and  guidance  continue  to  be 
established,  particularly  in  the  area  of  cybersecurity.  Agile  was  described  in  some  of  the  sources 
as  a  means  to  mitigate  cyber  threats  by  expediting  the  delivery  of  capabilities.  Numerous 
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challenges  were  identified  with  the  DIACAP  process  specifically,  such  as  the  lengthy 
accreditation  timeline,  unclear  reaccreditation  guidance  for  incremental  capability  releases,  and 
lack  of  collaboration  between  with  Agile  and  IA  teams.  However,  The  RMF  (DoDI  8510.01) 
does  not  explicitly  disqualify  the  use  of  Agile  in  the  DoD. 

Programs  regularly  use  integrated  master  schedules  and  EVM  (depending  on  contract 
type,  size,  and  duration)  to  monitor  contractor  performance.  Because  requirements  are  subject  to 
a  significant  amount  of  fluidity  in  Agile  projects,  programs  may  be  challenged  in  maintaining  an 
accurate  IMS,  or  using  traditional  EVM  to  track  performance.  While  not  all  programs  are 
required  to  use  EVM,  it  can  be  adapted  for  use  on  Agile  programs.  However,  the  process  would 
need  a  significant  amount  of  coordination  between  the  government  and  contractor  teams,  and  it 
could  be  labor  intensive  (Lapham  et  al.,  2010). 

The  Agile  methodology  makes  allowances  for  changes  in  requirements  throughout  a 
project’s  life  cycle.  One  benefit  of  this  level  of  requirements  fluctuation  is  that  capabilities  are 
allowed  to  evolve  based  on  user  needs.  However,  the  changes  introduce  difficulties  with 
managing  iterative  requirements  and  prioritization  by  end  users,  and  they  may  trigger  contract 
modifications  that  are  labor  and  time  intensive.  While  some  of  the  challenges  associated  with 
requirements  management  can  be  attributed  to  the  relatively  slow  pace  of  the  acquisition  and 
contracting  processes,  some  are  related  to  other  areas  such  as  culture,  end  user  involvement,  and 
training. 

A  common  conclusion  found  in  multiple  sources  was  that  a  primary  barrier  to  adopting 
Agile  in  the  DoD  may  be  cultural  (Lapham  et  al.,  2010).  Agile  is  not  a  blanket  solution  for  all 
DoD  IT  programs.  However,  it  is  a  viable  option  for  programs  that  are  able  to  shift  the  structures 
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within  their  programs  to  accommodate  a  more  streamlined  process  that  emphasizes  smaller, 
more  frequent,  capability  releases  (Modigliani  &  Chang,  2014). 
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Chapter  5  -  Interpretation 


The  intent  of  this  study  was  to  examine  the  extent  to  which  the  DoD  IT  acquisition 
framework  impedes  greater  adoption  of  Agile  IT  software  development  across  the  DoD. 
Historical  sources  were  evaluated  against  current  DoD  policies,  processes,  laws,  and  guidance  to 
identify  potential  obstacles  in  implementing  Agile  practices.  The  alternative  hypothesis  is  that 
the  DoD  acquisition  framework  impedes  Agile.  Based  on  the  findings  presented  in  the  previous 
chapter,  from  this  author’s  perspective,  the  original  hypothesis  has  been  invalidated.  This  chapter 
presents  interpretations  constructed  from  the  historical  review  and  comparative  analysis,  and 
provides  recommendations  for  possible  future  research  into  Agile  IT  software  development  in 
the  DoD  environment. 

Conclusions 

The  problem  statement  defined  in  this  study  was  that,  while  efforts  such  as  acquisition 
reform  and  BBP  have  emphasized  the  need  for  speed  and  flexibility  within  the  DoD,  lack  of 
agility  in  the  defense  acquisition  environment  is  still  identified  today  as  a  persistent  issue.  The 
rigid  and  laborious  nature  of  the  DoD  acquisition  process  was  noted  in  the  NDAA  (2015)  as  a 
key  impediment  to  delivering  timely  capabilities  in  the  DoD  environment.  This  study  aimed  to 
reveal  enduring  conditions  in  the  DoD  environment  that  continue  to  impede  further  Agile 
adoption.  Specifically,  this  study  addressed  the  following  question:  does  the  Agile  software 
development  methodology  complement  the  DoD  acquisition  process? 

The  investigation  revealed  that  the  primary  obstacle  to  increased  adoption  of  Agile 
practices  in  the  DoD  is  not  fundamentally  policy  or  process  related,  nor  unbendingly  restricted 
by  law  or  regulation.  The  DoD  5000  series  directives  and  instructions  actually  encourage 
flexibility  in  tailoring  program  oversight  and  documentation  requirements  to  reduce 
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administrative  bureaucracy.  In  reality,  the  principal  limitation  identified  in  multiple  sources  was 
largely  cultural.  Like  many  sources,  Modigliani  and  Chang  (2014)  identified  cultural  change  as  a 
critical  component  in  successfully  moving  the  DoD  into  greater  adoption  of  Agile.  The  authors 
supported  Agile  as  a  means  to  improve  capability  delivery  and  provided  insight  into  ways  that 
the  DoD  acquisition  framework  could  be  tailored  to  accommodate  effective  Agile 
implementation.  The  report  also  offered  detailed  recommendations  for  adopting  Agile  for 
specific  acquisition  functional  domains,  including  requirements,  systems  engineering, 
contracting,  cost  estimation,  metrics,  testing,  and  deployment/sustainment.  In  summary,  the 
authors  postulated  that  while  Agile  is  not  the  panacea  for  all  DoD  IT  programs,  it  is  a  feasible 
alternative  for  programs  that  are  able  to  embrace  Agile  practices  to  enable  a  streamlined 
development  approach. 

While  the  DoD  acquisition  framework  does  not  prevent  the  implementation  of  the  Agile 
software  development  methodology,  many  factors  may  present  challenges  in  doing  so 
effectively.  However,  based  on  the  findings  presented  in  several  sources,  these  challenges  can  be 
mitigated  if  those  who  are  implementing  Agile  practices  can  plan  ahead  and  adjust  accordingly 
(Modigliani  &  Chang,  2014). 

Acquisition  Oversight.  From  an  acquisition  oversight  perspective,  nothing  was  noted 
within  the  DoD  5000  series  directives  and  instructions  that  explicitly  restricts  the  use  of  Agile  IT 
software  development.  In  fact,  tailoring  of  oversight  and  documentation  is  encouraged  (DoD, 
2007).  Acquisition  program  models  were  developed  specifically  to  accommodate  incrementally 
structured,  software-intensive  programs  (DoD,  2017).  Flexibility,  responsiveness,  and  innovation 
are  all  policies  highlighted  in  DoDD  5000.01  (DoD,  2007)  and  are  considered  to  be  essential 
characteristics  of  the  Agile  methodology  (Lapham  et  al.,  2010). 
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The  root  of  the  concerns  identified  in  the  literature  review  may  simply  be  an  issue  of 
misinterpretation  or  limited  awareness.  Lack  of  communication  between  the  MDAs  and  PMs 
regarding  the  degree  of  tailoring  that  is  allowable  for  each  program  may  be  a  prime  source  of 
misinterpretation.  Additionally,  program  offices  may  not  possess  the  necessary  knowledge  and 
training  to  properly  streamline  individual  programs  and  implement  Agile  practices,  while  still 
complying  with  applicable  laws  and  policies. 

Contracting.  The  contracting  process  can  be  prohibitive  in  terms  of  required 
documentation  (i.e.,  CDRLs;  Lapham  et  al.,  2010),  collaboration  challenges  between  the 
contractor  and  government  (Modigliani  &  Chang,  2014),  lengthy  contracting  timelines,  and  the 
administrative  burdens  of  contract  modifications  for  any  changes  to  the  requirements.  This  can 
negatively  impact  a  program’s  ability  to  ramp  up  on  resources,  or  quickly  respond  to 
requirements  changes  when  needed  (GAO,  2012). 

The  FAR  is  certainly  a  complex  and  lengthy  document.  This  author  does  not  disagree 
with  the  assertion  by  some  sources  that  the  contracting  process  could  benefit  from  reform. 
However,  the  FAR  empowers  the  government  to  collaborate,  innovate,  and  use  sound  judgment 
to  determine  the  most  suitable  approaches  for  managing  their  contracts,  consistent  with 
applicable  laws,  regulations,  and  policies  (DoD,  2005a). 

The  fundamental  issue  in  the  contracting  area  is  two-fold.  First,  there  appears  to  be  a  lack 
of  Agile  knowledge  and  training  opportunities  in  the  contracting  community  (Wrubel  &  Gross, 
2015).  Second,  there  seems  to  be  a  lack  of  awareness  of  existing  guidance  and  best  practices 
related  to  executing  contracts  suitable  for  Agile  development.  Workarounds  can  be  leveraged  to 
reduce  timelines,  such  as  selecting  the  right  contract  vehicle  to  expedite  task  order  delivery,  and 
establishing  contract  vehicles  at  the  enterprise  level  (Modigliani  &  Chang,  2014).  However, 
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those  in  the  contracting  community  must  be  properly  educated  on  Agile  concepts  and  best 
practices,  and  work  collaboratively  with  programs  to  enable  an  Agile  approach. 

Cost  Estimation.  Based  on  the  historical  review  and  comparison  to  existing  DoD  policy 
and  guidance,  cost  estimation  and  realism  early  in  a  program  is  a  valid  constraint/challenge. 
DoDD  5000.01  provides  challenges  in  terms  of  requiring  life-cycle  cost  estimates  up  front.  The 
Program  Stability  Policy  (DoD,  2007)  requires  a  determination  of  the  system  design  before  a 
program  is  fully  funded.  These  conditions  do  not  necessarily  align  with  the  evolving  nature  of 
requirements/design  in  an  Agile  context,  as  noted  in  SEI’s  technical  report  (Lapham  et  al.,  2010). 

This  obstacle  can  be  overcome  if  programs  properly  coordinate  with  the  approving 
authority  (MDA)  to  provide  an  alternative  solution  that  still  meets  the  requirement.  For  example, 
to  address  the  Program  Stability  Policy,  Lapham  et  al.  (2010)  noted  that  Agile  can  accommodate 
an  overall  architectural  framework  to  a  final  system  design  up  front.  Additionally,  high-level  cost 
estimates  can  be  provided  early,  with  the  understanding  that  these  estimates  will  be  refined  over 
time  as  a  solution  is  iteratively  developed  (GAO,  2012). 

Information  Assurance.  IA  is  a  broad  area  in  which  policy  and  guidance  continue  to  be 
established.  DoDI  5000.02  only  recently  added  provisions  for  cybersecurity.  Several  sources 
identified  the  cumbersome  and  time-intensive  nature  of  the  DoD  acquisition  process  as  a 
roadblock  for  countering  cyber  threats  (Modigliani  &  Chang,  2014).  Agile  IT  software 
development  was  identified  as  a  potential  means  to  mitigate  cyber  threats  by  expediting  the 
delivery  of  capabilities.  On  the  other  hand,  expediting  software  development  could  potentially 
introduce  defects  that  leave  capabilities  vulnerable  to  attacks  (Schoeni,  2015).  IA  and 
cybersecurity  are  areas  that  should  be  explored  further  to  identify  potential  benefits  and  threats. 
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Program  Cost  and  Performance  Monitoring.  Requirements  fluidity  is  inherent  in 
Agile  development  projects,  introducing  complications  in  maintaining  an  accurate  IMS.  (Lapham 
et  al.,  2010).  Additionally,  EVM  is  not  applicable  for  all  contracts  (depends  on  type  and  value), 
but  can  present  challenges  for  Agile  development  programs  that  are  required  to  use  it.  Based  on 
the  findings  gathered  from  the  historical  review,  many  of  the  challenges  from  EVM  and  IMS 
accuracy  are  not  insurmountable  (Lapham  et  al.,  2010).  However,  EVM  challenges  are  not 
unique  to  Agile  programs.  This  author  speculates  that  EVM  policy  and  guidance  should  be  re¬ 
evaluated  for  all  DoD  programs  for  clarity  and  from  a  value-proposition  perspective. 

Requirements  Management.  As  mentioned  previously,  a  key  difference  between  Agile 
development  and  traditional  development  is  the  nature  by  which  requirements  are  defined.  In  a 
traditional  development  effort,  requirements  are  defined  and  essentially  locked  in  up  front.  In  an 
Agile  context,  requirements  are  allowed  to  evolve  over  the  course  of  the  project  life  cycle  and  as 
a  result  of  close  collaboration  with  stakeholders.  While  requirements  fluidity  can  create 
challenges  in  terms  of  contract  changes,  performance  monitoring,  schedule,  and  other  areas 
identified  in  Chapter  4,  there  are  also  benefits  to  an  iterative  requirements  process.  This  allows 
plans  to  adapt  based  on  user  need  and  without  creating  rework  (Welby,  2013). 

Based  on  the  findings,  the  DoD  acquisition  framework  does  not  restrict  the  use  of  an 
iterative  requirements  approach  inherent  in  the  Agile  methodology.  There  are  workarounds  to 
mitigate  challenges  associated  with  changing  requirements,  such  as  a  contract  strategy  that 
accommodates  this  condition  (see  previous  section  on  contracting).  The  DoD  5000  series 
directives  and  instructions  encourage  incremental  development  and  provide  a  framework  for 
software-intensive  programs.  The  JCIDS  model  and  IT  Box  Model  also  accommodate  an 
iterative  approach.  The  ICD  reduces  the  documentation  requirement,  which  can  be  broken  down 
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into  a  series  of  requirements  definition  packages  to  capture  smaller,  more  discrete  scopes  of 
work  (Modigliani  &  Chang,  2014,  p.  21). 

Recommendations 

The  DoD  should  place  greater  emphasis  on  providing  the  necessary  education,  training, 
and  guidance  to  members  of  the  DoD  community  involved  in  enabling,  executing,  and 
overseeing  Agile  development  initiatives. 

This  author  also  recommends  that  the  DoD  leverage  industry  analyses  and  best  practices 
wherever  practicable  to  understand  key  considerations  before  attempting  to  implement  an  Agile 
approach. 

The  DoD  has  expended  considerable  energy  toward  reforming  the  acquisition  process, 
and  organizations  continue  to  execute  Agile  software  development  programs.  Future  research  is 
encouraged  to  evaluate  the  extent  to  which  the  DoD  is  ultimately  successful  in  its  efforts  to  inject 
greater  agility  into  how  capabilities  are  developed  and  delivered.  Such  research  should  generate 
additional  lessons  that  can  be  used  to  improve  Agile  implementation  in  the  future. 

Limitations  of  the  Study 

The  scope  of  this  research  was  initially  limited  to  accessible  literature  from  government 
sources  and  industry  partners  who  adopted,  or  assessed  the  practicability  of  adopting,  Agile  IT 
software  development  in  the  DoD  environment.  However,  the  amount  of  available 
documentation  in  this  area  was  underestimated.  Additional  sources  continued  to  be  discovered 
over  the  course  of  this  study.  In  the  interest  of  time,  not  all  sources  could  be  considered, 
introducing  the  likelihood  that  other  critical  factors  affecting  Agile  implementation  in  the  DoD 
were  not  identified. 
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The  interpretations  provided  in  this  study  are  based  on  the  assumption  that  the  references 
were  credible  and  not  overly  biased  in  terms  of  the  DoD  acquisition  process  or  Agile  software 
development.  There  was  no  opportunity  to  interview  the  authors  for  clarification,  or  to  validate 
this  author’s  comprehension  of  their  findings. 

The  ultimate  goal  of  this  study  was  to  invoke  further  dialogue  about  the  effectiveness  of 
acquisition  process  and  the  DoD’s  efforts  to  reform  it.  Through  this  qualitative  investigation, 
historical  perspectives  on  potential  barriers  for  Agile  in  the  DoD  were  evaluated  against  the  most 
current  DoD  policies,  laws,  regulations,  and  guidance  available.  Based  on  the  findings,  this 
author  concludes  that  the  DoD  acquisition  process  does  not  fundamentally  impede  Agile 
adoption.  Many  of  the  challenges  associated  with  adopting  Agile  are  not  specific  to  the 
acquisition  framework.  A  primary  challenge  can  be  attributed  to  culture.  There  appears  to  be  a 
hesitancy  to  adopt  a  development  approach  that  is  not  widely  understood  in  the  DoD  community 
and  is  counter  to  the  traditional,  waterfall  methodology.  Granted,  the  DoD  acquisition  framework 
is  sorely  in  need  of  reform,  and  it  can  prove  to  be  overly  rigid  in  some  cases.  However,  there  are 
aspects  of  DoD  policy  and  guidance  that  complement,  and  facilitate  implementation  of,  Agile 
software  development.  While  some  areas  such  as  cost  estimation,  the  traditional  structure  (and 
documentation  requirements)  of  the  milestone  review  process,  and  contracting  can  present 
challenges  and  potential  constraints,  many  of  these  obstacles  can  be  overcome  with  the  proper 
planning  and  open  communication.  Those  in  the  acquisition  community  and  their  industry 
partners  must  work  collaboratively  to  address  potential  barriers  up  front  and  early  to  ensure  that 
implementing  an  Agile  framework  is  successful. 
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Glossary  of  Acronyms  and  Terms 


ACAT . acquisition  category 

AT&L . Acquisition,  Technology,  and  Logistics 

CIO . Chief  Information  Officer 

CDD . Capability  Development  Document 

CDRL . contract  data  requirements  list 

CPD . Capability  Production  Document 

DAU . Defense  Acquisition  University 

DIACAP . Department  of  Defense  Certification  and  Accreditation  Process 

DoD . Department  of  Defense 

DoDD . Department  of  Defense  Directive 

DoDI . Department  of  Defense  Instruction 

EVM . earned  value  management 

FAR . Federal  Acquisition  Regulation 

GAO . Government  Accountability  Office 

IA . information  assurance 

ICD . Initial  Capabilities  Document 

IMS . integrated  master  schedule 

IT . information  technology 

JCIDS . Joint  Capabilities  Integration  &  Development  System 

JROC . Joint  Requirements  Oversight  Council 

MDA . Milestone  Decision  Authority 

NDAA . National  Defense  Authorization  Act 
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PLCCE 


program  life  cycle  cost  estimate 


PM . program  manager 

USAF . United  States  Air  Force 

USD(AT&F)  ..Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Fogistics 
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